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Abstract 

Problem statement: In art education, one of the most difficult tasks is to 
evaluate the artistic works of students. Portfolio assessment is a method of 
assessment that is commonly used in artistic education. The major research 
question investigated in the study is: "What are the advantages and 
disadvantages of digital portfolio in an art classroom?" 

Purpose of the study: The first aim of this study is to investigate digital portfolio 
assessment in higher art education with the combination of self, peer and 
instructor ratings. The second aim is to determine opinions and perspectives of 
the students on the self and on peer and instructor assessment. 

Research method: This research contains data collected from 34 students. The 
students were 14 males and 20 females. Age levels of the students ranged 
from 21 to 28. The students were asked to prepare their portfolio in the 
computer environment. The students were given 4 weeks to prepare the 
digital portfolios. The digital portfolios were evaluated as to the rubric 
which was previously prepared and adopted by the students and 
researcher. The students first evaluated their own portfolios and then those 
of their peers. Finally, the researcher evaluated all of the portfolios. 

Results and discussion: Regarding the evaluation of the results, the lowest 
correlation values were found between the instructor and the peer, and the 
highest correlation values were found between self and the peer. In the 
interviews with the students, the students judged the peer assessment to be 
less trustworthy than instructor assessment. The hesitation of the students 
about self assessment and the peer assessment might be caused by Turkish 
culture, because an instructor as a superior authority figure is always 
considered the most trustworthy person by the Turkish students. 
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Recommendations: In terms of educational applications, digital portfolio as 
an assessment tool should be improved and widely applied in classroom 
settings where the subjects especially deal with artistic art works. In art 
teacher training programs, the importance of triadic assessment should be 
seriously taken into account. This would be vitally important to teach how 
viable criteria can be set up for the assessments. 

Key words: Art education, portfolio assessment, digital portfolio, assessment rubric. 

Introduction 

Portfolios are used in many fields such as fine arts, marketing, architecture, and 
education. A portfolio is defined as "a purposeful collection of student works that 
reflect the efforts, development and successes of the learner" (Jacobson, Sleicher & 
Maureen, 1999; Mullin, 1998; Paulson, Paulson & Meyer, 1991, p. 60; Stiggins, 1997). 
Portfolio assessment procedures are increasingly recommended to document 
students' performance in school and in extra-curricular activities. During the past 
two decades, the use of portfolios to assess creativity and giftedness has been widely 
accepted due to the development of explicit, well-defined procedures for their use 
(Johnsen & Ryser, 1997). Examples of the students' work are typically reviewed by 
knowledgeable individuals (Davis, 1997). Authentic measures such as portfolios, 
work samples, and biographical information are essential components in identifying 
talent, for example, in the visual arts. 

Properties of portfolio assessment. The most distinctive property of a portfolio makes 
a learner both assessor and assessed. In this case, apart from being the object of the 
assessment, the student is both the partner of the assessed object and the assessment 
(Wolf, 1991, p. 130). Here, the learner actively participates in the selection of the content 
and deterrnining the selection of the criteria. Therefore, portfolios serve both for teachers 
and for students. They not only provide an opportunity for students to project their 
successes, but also provide an opportunity for teachers to evaluate the development and 
success of their students. Students test their own works and project them on their targets 
for the future (Stiggins, 1997). Traditional tests do not reveal the development and all the 
skills of the individual (Barton & Collins, 1993). 

Benefits of portfolio assessment. One of the many benefits of a portfolio brings 
clarity to the fairness in assessing students' performances. By means of portfolio 
assessment, assessment is not a secret method. Students judge the quality of their 
own works and develop standards. In determining the assessment criteria, student- 
student negotiation and student-teacher negotiation play important roles. In order to 
shape the assessment criteria, students can be allowed to have a class discussion. 
This method provides an educational environment both for the students and the 
teachers (Mullin, 1998). This environment enables the students to be responsible for 
their own development because it contributes to the activation of the meta-cognitive 
awareness to monitor their own learning. So, a portfolio assessment is not simply a 
tool to use to evaluate the end product. It also monitors students' learning processes. 
However, it is important to note that an assessment rubric should be reliable and 
applicable as independently as possible. Students should reach a common decision 
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by acting together to determine the structure, contents and criteria of the portfolio 
and the necessary documents under the guidance of their instructor. 

Triadic assessment. Some form of self and peer assessments in higher education 
have been documented in the literature (McConnel, 2002). Gale, Martin and McQueen 
(2002) developed a triadic {self, peer and tutor) assessment by using questionnaire and 
semi-structured interview techniques at the University of Plymouth in the UK. The 
research sample was drawn from groups of students studying at different stages of the 
Bachelor Education and Training program within the Faculty of Arts & Education. 
Questions of power and knowledge were closely examined, alongside a close critical 
analysis of the way in which the professional identity and assessment practice style of 
those involved in the research were influenced by the way they were situated within a 
variety of different practice contexts. However, researchers suggested that triadic 
assessment was not a transparent activity to easily generalize due to its lack of 
universal features and characteristics. 

A similar research was conducted by Centra (1994). Centra examined the 
correlation between scores given by two peer faculty members and four deans to the 
same portfolios. In the portfolios, faculty documented their accomplishments and 
wrote personal statements in four areas: teaching effectiveness, service to the college 
and community, personal credentials, and professional activities. Results of Centra's 
study indicated that all groups of raters judged the portfolios with very high scores. 
The deans gave the lowest ratings for total teaching effectiveness. Centra realized 
that a problem with the portfolio assessment in his study is that no standard criteria 
was used to rate the portfolios. Centra suggested that when using portfolios for 
evaluation, a standard criteria by which they are to be judged should be established. 
Usage of Portfolio Assessment in Art Education 

Eisner (1985) developed a holistic approach, which can deal with both processes 
and products, to characterize the ways that professional assessors perform tasks 
similar to those of connoisseurs and critics in the arts. Such a holistic assessment 
approach is certainly possible for much educational judgment, but it must go beyond 
the traditional assertion. 

According to Madeja (2004), appropriately trained art teachers are capable of 
judging the aesthetic merits of artwork by using well established techniques in the 
field. However, standardized tests should not be the only measure, especially in arts 
assessment. In art education, a portfolio assessment is different from the traditional 
assessments in terms of scoring. Multiple parties such as the instructor, artists and 
students could participate in rating a portfolio, but it is important to have an 
agreement in scoring when the parties rate the portfolios independently. Sabol (2006) 
conducted a study related to portfolio assessment rated by the art teachers, art 
students, and artists. Sabol has found impressive results in his study. Fifty-nine 
elementary, middle, and secondary art teachers, 472 students, and 50 artists were 
involved in this research. Even though the criteria used to assess the portfolios 
suggest different priorities for the art teachers, art students, and artists in his study, 
high levels of agreement were found between the art teachers and students in scoring 
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the portfolios. Sabol interpreted this as a possible result from art teacher imposed 
criteria for artwork (p. 10). 

The use of digital portfolio assessment in art education. The use of computers in art 
education is increasing day by day. While art teachers do not doubt that digital 
portfolios have advantages, they are skeptical about the reliability of using digital 
portfolios in order to evaluate students' artwork. They want to know whether 
evaluations of digital reproductions of artwork would be comparable to evaluations of 
actual artwork. Furthermore, they are concerned as to whether evaluations of portfolios 
by art educators would be comparable and reliable (Dorn, Madeja & Sabol, 2004). 

In a recent study, Dom and Sabol (2006) carried out a research involving 178 
students of 29 secondary school teachers from four school districts in Florida and 
Indiana by using a digital portfolio. The results suggested that electronic portfolios 
could be used to reliably estimate students' art performances and scores, because art 
teachers' evaluations of the digital copies of the actual artwork were consistent with 
the evaluations of these same works in the actual form. 

The Purpose of the Study and Research Question 

The literature on digital portfolio assessment suggests that more research is 
needed that examines the effectiveness of this type of assessment and that reveals its 
advantages and disadvantages. Therefore, the main aim of this study is to investigate 
digital portfolio assessment in higher art education with the combination of self, peer 
and instructor ratings. In addition, the correlation among ratings of the self, peer and 
instructor on digital artwork is an interest of this study to judge the reliability of this 
technique. The major research question investigated in the study is: What are the 
advantages and disadvantages of digital portfolio in an art classroom? 

Research Method 

Participants 

The research involved the fourth year (senior) students in the art teacher training 
program at a middle-sized Turkish University. The program aims to train art 
teachers for primary and secondary school levels with four-years of study. Turkish 
art teachers are trained according to the "Pre-service Art Teacher Training Program 
(ATTP)" by Kiri§oglu and Stokrocki (1997). In the ATTP, the first three years mainly 
focus on developing trainees' art skills, such as art history, media applications, 
aesthetics, art criticism, computer operations, and drawing-painting skills. Starting 
with the third year, teacher training courses (i.e., classroom management, teaching 
methods, and lesson plans) are provided. The last year of the program emphasizes 
the practicum approach in school settings; where trainees are taken to schools to 
teach lessons and understand the daily routines in school settings. 

The study sample consisted of 34 senior students at an art teacher training 
program at a College of Education, during the spring semester of the 2005-2006 
academic year. The participants in the study were 14 males (41.1%) and 20 females 
(58.8%). Age levels of the participants ranged from 21 to 28. The average age for 
students was 21.71 years old. 
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Data Gathering Method 

Both qualitative and quantitative data were utilized in the study. The following 
sections will clarify the sources of the data and their analysis. 

I. Quantitative data, in this study, the students and the researcher utilized the same 
assessment rubric in order to assess the digital portfolio materials of the students. The 
criteria for the selected rubric were adopted from http://www.bcpl.net/~sullivan 
/modules/tips/rubrics_sec/ scrapbook.html (Table 1.). Validity of a portfolio assessment is 
supported in two ways: (1) a well-established appraisal system, which encourages 
self-development, provides the opportunity for creative problem solving, and (2) this 
system focuses on individual effort for accepted program goals (Bursch, 1997). 
Therefore, the basic components of the student performance required for scrapbook 
preparing were: "content, theme, language & conventions, and overall effectiveness." Point 
assignment was used for each component of the assessment rubric. Points scale 
ranged from "two" to "eight.” This kind of point assignment provides flexibility for 
the assessors in scoring (Custer, 1996; Herman, Gearhart & Baker, 1993; Moscal, 2000; 
Sabol, 2006). The scoring criteria were as follows: Minimal 2 point, Basic 4 point. 
Good 6 point, and Perfect 8 point. 


Table 1 

Scrapbook Rubric* 



Perfect 

Good 

Basic 

Minimal 


8 

6 

4 

2 


Information is 

Main points 

Some main 

Main points 


complete and is 

are covered but 

points and details 

are not complete 


enhanced by accurate 

lack some detail. 

are missing. More 

and are greatly 


and appropriate 

Number and 

and better visuals 

lacking in detail. 


details. Pictures, 

types of visuals 

could be used; 

Very little 


photographs. 

are adequate as 

captions only 

pictorial 


drawings, diagrams. 

are captions. 

identify and label 

representation is 


graphs, or other 

Design elements 

rather than 

present; captions 


similar devices add to 

and principles 

explain. Shows 

are incomplete. 


the overall 

used are 

evidence of use of 

There is no 

c 

effectiveness of the 

adequate. The 

some design 

consideration of 

a; 

g 

scrapbook; captions 

minimum 

elements and 

design elements 

o 

U 

are relevant and 

number/ types of 

principles. 

and principles. 

explanatory. Space, 

sources are 

Sufficient 

The list of sources 


shapes, textures, and 

present and are 

number/ types of 

if inadequate in 


colors provide 
information 
themselves and add to 
the overall 
effectiveness of the 
scrapbook. All sources 
are properly and 
thoroughly cited; the 
number /types of 
sources are exceeded. 

cited properly. 

sources are 
lacking; citations 
are not all 
formatted 
properly. 

number/ types 
and format of 
items. 
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Theme 

There is 

wholeness about the 
scrapbook; the theme 
is consistent 
throughout. The cover 
clearly identifies the 
theme. 

Most of the 
information 
relates to the 
theme of the 
scrapbook. The 
cover is relevant 
to the contents. 

Only a 

portion of the 
information 
relates to the 
theme of the 
scrapbook. The 
cover is unclear in 
its message. 

Confusing 

and/or 

inconsistent 

information. 

Language & 
Conventions 

Conventions of 
spelling, punctuation, 
and grammar are used 
with a high degree of 
accuracy. 

Most 

conventions of 
spelling, 

punctuation, and 
grammar are 
used accurately. 

Common 
conventions of 
language are used 
with some 
accuracy but there 
are some 
mistakes. 

A significant 
number of errors 
are made in 
spelling, 

punctuation, and 
grammar. 

Overall 

Effectiveness 

The requirements 
of the assignment 
have been exceeded. 
The scrapbook is very 
creative and 
interesting. 

All the 

requirements of 
the assignment 
have been 
fulfilled. The 
scrapbook is neat 
and presentable. 

Only some of 
the assignment 
requirements are 
fulfilled. Areas of 
the scrapbook 
lack neatness. 

Few of the 
assignment 
requirements 
have been met. 
The presentation 
as a whole lacks 
neatness. 


*The rubric adopted from http:/ /www.bcpl.net/~suUivan/modules/tips/rubricssec/ 
scrapbook.html 


The process of preparing the digital portfolios was completed as follows: First of 
all, students were required to identify the aims for their portfolios they would 
prepare. In the first week, it was carried out into the identification of the aims. The 
significance and necessity of the aims, this frame, which would be provided for the 
portfolios the students would improve, were explained to the students. 

Identification of the necessary features for a qualified scrapbook and the 
presentation of the assessment rubric were in the frame. The features that would be 
necessary for a qualified scrapbook were determined by face-to-face discussions with 
students. The assessment rubric which had been improved beforehand was 
presented in the frame features. These frame features highlight the major 
components for a qualified scrapbook that would be carried out by students. Also, 
the duties and the rubric indicating the criteria for evaluation and scoring were 
explained to the students in detail. 

The scrapbook examples and the rubric were sent to the e-mail addresses of the 
students. The paper-based samples and the rubric were given to the students in the 
classroom. Thus, the students had an opportunity to know not only which samples of 
the work were good, average and unsuitable but also for what reasons those works 
were evaluated in that way. They were given the chance to improve their works in 
that frame. 

Each student was required to prepare at least four scrapbooks for his/her 
portfolio that s/he would hand in at the end of the research. Each scrapbook contains 
a well-known artist's paintings and bibliography. In the instruction, it was pointed 
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out that the selected artists in the scrapbooks should be from different time periods, 
from the renaissance era to the present-day. Students also had freedom to prepare 
additional scrapbooks. Students had 4 weeks to turn in their scrapbooks. With 4 
hours in a computer-based environment for the students, a total of 8 hourly practices 
were carried out each week. During the research session, the students made use of 
various web pages and software package programs such as Word, Paint Brush, 
Power Point Presentation and illustrator, and Photoshop to prepare their digital 
portfolios. 

This course was not limited to classroom practices and face to face discussions. 
Discussions were also taking place on the online environment to extend their 
learning. Although we did not use an online discussion board, students asked their 
questions, presented their works, and shared their ideas with their classmates via 
electronic mail and msn messenger. In this way, each student had an important 
opportunity to hear the critiques of the others for their works. In each stage of the 
works (at the end of each course) the quality of the students' works were evaluated. 
In addition, necessary corrections and feedback were provided in order that there 
was a scope for students to take note of incorrect and inefficient parts of their works 
and to improve on them. 

The explanation for the storage of the students' works and how they would 
present prepared portfolios was given in the last week. There were some alternatives 
for the students to store their works on computers, in electronic mail addresses, or on 
CDs, DVDs or flash disks. Students were permitted to use a scanner in order to 
transfer the works done by pen and paper to the digital environment. Students 
presented the drafts of each of their own works in a chronological order. 

The students (self and peer) and the researcher participated in the assessment. 
Each student first evaluated his/her own portfolio according to the rubric and then 
each student evaluated the portfolio of his/her friend according to the rubric. In peer 
assessment, the students did not know whose portfolio they were evaluating (blind 
review). Finally, the researcher evaluated all portfolios. The researcher has expertise 
in art education and the use of portfolio assessment in educational settings. The 
assessors made individual assessments without external influences. In addition, the 
assessors didn't do a different evaluation for digital portfolios. In other words, the 
assessors made an assessment with respect to the assessment rubric which was 
developed for students' digital portfolios. 

Analysis of quantitative data: After collecting the scores given by the three parties— 
self, peer and the researcher— at the end of the research, Pearson correlation 
coefficients were calculated by using the SPSS 13.0 statistical package program. The 
significance levels of the coefficient values were determined according to Hopkins 
(1997). The coefficient values of 0.10-0.30, 0.30-0.50, and 0.50-0.70 are considered low, 
medium, and high correlations, respectively. 

II. Qualitative data. After the evaluation procedures were completed for the 
portfolios, several open-ended and semi-structured questions were asked to the 
students. The examples from the questions sets are as follows: What do you think 
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about that evaluation procedure overall? What kind of software did you prefer when 
you prepared your portfolio? Why? To what extent were you confident as you 
evaluated your own portfolio? Why? To what extent were you confident as you 
evaluated the portfolio of one of your friends? Why? What do you think about this 
triple evaluation? What are your overall experiences from all of these? The 
interviews were audio-taped and then transcribed, in the analysis, after the data was 
encoded, the themes were found and then arranged (Strauss & Corbin, 1998). Finally, 
each theme was expressed with frequencies and each was interpreted. 


Results and Discussion 

Evaluation of Statistical Findings 

in the correlation analyses, while the lowest correlation was .19 between the 
researcher and peer in scoring the Language & Conventions, the highest correlation 
was found to be 0.50 between self and peer ratters in scoring the Overall Effectiveness 
and in scoring Theme (See Table 2). Compared to the ranges of values determined by 
Hopkins (1997), the values in Table 2 can be accepted as at the medium level. Also, 
Cohen and Lea (2003) stated that the value at the level of 0.30 was a medium level 
value for social sciences. 


Table 2 


The Correlations among the Assessors (R: Researcher, S: Self, P: Peer) 


Sub-Components 

Assessors 

(R,S,P) 

R 

S 

P 


R 

1.000 



Content 

S 

0.491* 

1.000 



P 

0.420* 

0.334* 

1.000 


R 

1.000 



Theme 

S 

0.502* 

1.000 



P 

0.389* 

0.323** 

1.000 


R 

1.000 



Language & 
Conventions 

s 

0.462* 

1.000 


p 

0.193** 

0.499* 

1.000 


R 

1.000 



Overall Effectiveness 

S 

0.467* 

1.000 



P 

0.372* 

0.504* 

1.000 


*p< 0.01, **p< 0.05 significance 

it was found that the highest correlation was between peer and self scoring 
(0.504) and the lowest correlation was between the researcher and peer (0.193). 
However, Sabol (2006) found greater levels of agreement in rankings which were 
found amongst the art teachers and students. Sabol interpreted this as possibly being 
the result of art teacher imposed criteria for artwork. The art teachers must be 
concerned with developing a range of knowledge and skills among their students. 
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According to Sabol, students in school focus on what art teachers teach. Students at 
home focus on skills with media and personal satisfaction derived from their art. 

Pitts, Coles and Thomas (1999) found the reliability of individual assessor's 
judgements (i.e., their consistency) as moderate, but inter-rater reliability did not 
reach a level that could support making a safe summative judgment. They said, 
"What can we learn from qualitative approaches? Qualitative research takes an 
interpretive, naturalistic approach to its subject matter; qualitative researchers study 
things in their natural settings, attempting to make sense, or interpret, phenomena in 
terms of the meanings that people bring to them" (p.519). In another research, Pitts, 
Coles, Thomas and Smith (2002) found an agreement between assessors ranged from 
"slight" to "fair." A kappa coefficient of 0.50 reflects the moderate agreement 
between assessors. 

Evaluation of Interviews and Observations 

From a grounded theory approach to the analysis of the data (Strauss & Corbin, 
1998), five broad analytic categories were built under which the experiences and 
opinions of students can be considered. The categories were related to the 
preparation of the digital portfolios and assessment processes. The features used in 
forming scrapbooks in the digital portfolio were taken into consideration in the 
categorization. The frequencies and percentages of the observed themes among 
students are presented in Table 3. The categorizations of the themes were based on 
the analyses of the interviews and observations of the researcher about the 
preparation and evaluation phases of the digital portfolios. The sums of the 
percentage in the Table 3 are not 100% because a majority of the students meet the 
criteria for more than one category. 


Table 3 


Frequencies and Percentages of the Categories 


Categories 

f 

% 

Approval of the researcher assessment 

29 

85.2 

Paucity of web pages 

28 

82.3 

Hesitation of the peer assessment 

25 

73.5 

Difficulty of computer software 

23 

67.6 

Hesitation of the self assessment 

21 

61.7 


Approval of the researcher assessment. For students, it sounds normal if the only 
assessor is the trainer. 85.2% of the students thought in that way but they did not 
have such a confident for themselves and their peers. This might be because they saw 
themselves as inexperienced in assessment. Some students such as student A, a male 
student, expressed that if the trainer (researcher) also participated in the assessment, 
then it was okay. 
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Student A- "1 was happy when I found out three persons will grade one portfolio. So, you 
will also give points, too. . . Then, there is no problem ..." 

In fact, self assessment and peer assessment were very strange to the students 
because they had gotten used to trainer or instructor assessment over the years. 
Thus, their hesitations and avoidances for the self and the peer assessments can be 
considered as a normal consequence. 

Paucity of web pages. The students stated that they easily obtained the paintings 
of the artists from the web pages; on the other hand, they didn't find enough 
information about the artists and their paintings. 82.3% of the students complained 
about the insufficiency of the web pages. However, some students like student B 
weren't very constrained in developing content since they prepared the scrapbooks 
with the paintings from the art magazines. 

Student B- "I could not find detailed information about Paul Gaugiun on the web pages. 
There were a lot of samples from his works, but there were little information related to his 
pictures. So, I used the art magazines and scanned his pictures and transferred them to the 
digital environment. Hope you have not got angry with mel..." 

Hesitation of the peer assessment. The students first objected when they were 
asked to assess the portfolios of their friends. It was observed that they did not want 
to take responsibility. They were told they would be teachers in the future and they 
would have to assess and grade their students' works. 73.5% of the students thought 
that peer assessment might be distrustful. Similar findings were found in a research 
of Gale et al. (2002). The students hesitated to evaluate their friends' portfolios. For 
example, student C, a female student, did not think that she could be objective in 
assessing someone's work. 

Student C- "Sir, you want me to assess my friend’s portfoliol... Shall I know whose 
portfolio I will be assessing?. I am not sure if I will be able to evaluate it objectively." 

With this in mind, student C has in fact accepted that one of the dimensions of 
the assessment is objectivity. Although she knows the criteria for the assessment, she 
has low self-confidence. 

Difficulty of computer software. The classroom observations let the researcher 
identify some significant problems. The students first tried to learn how to use 
computer software that they need for preparing a scrapbook. The four weeks given 
to the students for their assignment was not a long period of time, because some 
students apparently also needed to develop specific skills to use the necessary 
software. Therefore, specific software skills needed to be introduced and practiced. 

The software popular among the students to prepare the digital portfolios were 
Paint Brush, Microsoft PowerPoint, and Word Processor. These softwares were seen as 
somewhat familiar and generally available on all computers. However, it was 
difficult for the students to use some package programs like Illustrator and 
Photoshop. Such difficulties were expressed by 67.6% of the students. Also, 
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observations of the researcher in the classroom supported such difficulties 
experienced by the students. An interview segment of student D, a female student, 
exemplifies how students had a hard time with the software. 

Student D- "At the beginning, the idea of preparing a scrapbook seemed great to me; 
however, when I sat at the computer, the first thing that came to my mind was to make a 
search over the internet. But I could not use a software by which I am able to prepare my 
work. It is very difficult to use the Photoshop software which you have recommended, so I 
preferred to use an easy software." 

It was noticed that there were differences between male and female students 
regarding the difficulty that they experienced in using the software. While 90.0% of 
the female students expressed some difficulty in using the software programs, only 
35.7% of the male students expressed similar difficulties in using the software. 
Female students frequently preferred to use Paint Brush, Power Point and Word 
Processor. Male students showed more effort in using the computer software. Male 
students used some of the software that female students used too; however, some 
male students were able to figure out how to use some difficult software like 
Photoshop. Some male students used Photoshop to arrange the dpi resolutions and 
sizes of the pictures that they found from the internet. For example, student E, a male 
student, expressed that although Photoshop is professional software, it is not 
necessary to be competent in it for their assignments. 

Student E- "When you said we can use the Photoshop software, I was astonished. As far 
as I know, it is a professional software program; however, all I needed to enlarge and reduce 
the sizes of the pictures. It was difficult at the beginning but I think I have overcome it. . ." 

Student E said that Photoshop was professional software; however, he figured 
out simple functions of Photoshop to prepare his own work. Although he found it 
difficult to use this software at the beginning, he overcame that problem; however, 
female students made no attempt to use the Photoshop software. 

Hesitation of the self assessment. The researcher received interesting responses 
when students were asked their opinions about their own works. As the students 
were only familiar with the traditional evaluation, it was very interesting for them to 
evaluate their own works. 61.7% of the students thought that their evaluation might 
be wrong so it would be more suitable if 1 do the evaluation. A female student, 
student G, gave an interesting response when the researcher asked how she felt 
about the self assessment. 

Student G- "Sir, why do you have us make the evaluations?...! don’t believe I can 
evaluate my own study in a truly way even if it is done according to the rubric! You give us 
too much power. Keep some of them in your own hands..." 

As you notice. Student G felt that she was given too much power for the 
evaluation because she thought that the evaluations should be tmder the heavy 
control of the instructor. Her thoughts might be an indication that she is very loyal to 
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the authority and that she accepts authority without questioning it at all; however, 
most students had already had this thought, too, before the start of self assessment. 
However, their worries and hesitations disappeared after the self assessment, 
because they had an opportunity to judge their own study. Mullin (1998) says that 
the most important benefit of portfolio assessment is the opportunity for the student 
to assess his own work. 

A Theoretical View to the Findings 

On the basis of the presented data, a model was generated to determine the 
students' perspectives on digital portfolio assessment (Figure 1.). 



Figure 1. Integration of the findings 


At the left of the diagram, there was the DIGITAL PORTFOLIO prepared on the 
COMPUTER where the ASSESSMENT was made. In the preparation of the digital 
portfolios, the students benefited from the SOFTWARE and WEB PAGES. Based on 
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the data taken from the students, the software fall into two categories: difficult and 
easy. The easy ones were PAINT BRUSH, POWER POINT and WORD PROCESSOR. 
The difficult software included ILLUSTRATOR and PHOTOSHOP. While Paint 
Brush, Power Point and Word Processor were preferred by FEMALE STUDENTS, 
Paint Brush and Power Point were constantly preferred by MALE STUDENTS. 
Photoshop, which was difficult software to use, was used by some male students. No 
student used Illustrator. The students benefited from the web pages in the creation of 
digital portfolios. While students found many PAINTINGS and PICTURES on web 
pages, they were not enough in terms of CONTENT. Some male students transferred 
the pictures they took from web pages to Photoshop. Some students scanned the 
pictures and contents of ART MAGAZINES and they placed them in the computer 
environment. 

In the second part of the diagram, there was the evaluation of the digital 
portfolios on the COMPUTER, ASSESSMENT. This assessment was made according 
to the rubric. OBJECTIVITY was important in assessment. In this assessment, 
INSTRUCTOR ASSESSMENT was more important. SELF and PEER ASSESSMENT 
was more trustworthy. There was low correlation between instructor assessment and 
peer assessment, and medium correlation between peer assessment and self 
assessment. The average values of all three assessments determine the score of the 
digital portfolio. 


Conclusion 

It is important to prepare a rubric in the beginning in terms of harmony amongst 
self, peer and the researcher's points. A lower correlation coefficient, reflecting less 
than moderate agreement, is unlikely to be acceptable. According to Pitts et al. (2002), 
whether a coefficient of 0.50 is acceptable is open to debate. Certainly, the higher 
value is the better. Nevertheless, these results show that discussion between 
assessors increases reliability above the levels often achieved in assessments of 
professional competence, and is an improvement on individual assessment. This 
result is also a result recommended by Centra (1994); however, it was determined the 
score points in the rubric ranged from 2 to 8. If it was arranged in way that the score 
points in the rubric ranged from 1 to 4, correlation values amongst the pointers might 
have been higher. The hesitation of the students about self assessment and peer 
assessment might be caused by Turkish culture, because an instructor as a superior 
authority figure is always considered the most trustworthy person by the Turkish 
students. Korkmaz and Kaptan (2003) suggested the practical information to the 
teachers on portfolio assessment both in their pre-service and in-service education 
should be given. Even though portfolio assessment has many difficulties, the use of 
portfolio assessment in our schools and teacher training systems should be taken into 
consideration. Because the portfolio assessment approach has better features than the 
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traditional assessment tools with regard to reflecting the efforts, development and 
successes of the students 

The capabilities of the students to justify their own works support their learning. 
Peer assessment can enable to the students a fair and an honest pointing habit; 
however, this evaluation in this study is not a generalizable one. Students from 
different training fields and age groups can reveal different opinions. Gale et al. 
(2002) stated that "triadic assessment was clearly not a transparent activity that 
revealed easily generalizable and universalizable features and characteristics." 

As a result of the current study, it is suggested that digital portfolio as an assessment 
tool should be improved and widely applied in classroom settings where the subjects 
especially deal with artistic art works. In the art teacher training program, the importance 
of triadic assessment should be seriously taken into accotmt. It is important to learn how 
to define assessment criteria. Also, it would be effective for Turkish students to overcome 
their cultural barrier regarding peer and self assessment types. Furthermore, it is 
suggested that the assessment exercises employed in this study would contribute to the 
art teacher candidates' professional development. In fact, the Turkish Ministry of 
National Education suggests as assessment tools the self assessment, group assessment, 
peer assessment, checklist, portfolio assessment and performance assessment in art 
education (MEB, 2006, p. 100). 

Limitations of the Study 

This study was conducted in the field of art education. Digital portfolios 
prepared with scrapbooks by the students were evaluated. Different results could be 
achieved from the data collected from different cultures' students. In this study, the 
students encountered such an assessment for the first time. Different results could be 
achieved with the students who have already been made familiar with such an 
evaluation. 
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Sana! Egitiminde Akranin, Kendisinin ve Ogreticinin 
Degerlendirmesi ile Dijital Siire^ Dosyasimn Bir Uygulamasi 

(Ozet) 

Problem DurumuSxaeq dosyalari birgok alanda kullanildigi gibi egitim 
alanmda da kullanilmaktadir. Stireg dosyasi gibi gali§ma ornekleri gorsel 
sanatlarda bireyin yeteneginin ve geli§iminin belirlenmesinde onemli role 
sahiptir. Stireg dosyasimn en ayirt edici ozelligi ogreneni hem 
degerlendiren hem de degerlendirilen yapmasidir. Ogrenci degerlendirme 
olgiitlerinin ve igerigin belirlenmesine aktif olarak katilir. Boylece, stireg 
dosyasi hem ogrencilere hem de ogretmenlere hizmet eder. Stireg dosyasi 
degerlendirme sanat egitiminde yaygm bir §ekilde kullanilan bir 
degerlendirme §eklidir. Sanat egitiminin en zor i§lerinden birisi 
ogrencilerin sanat gali§malarinm degerlendirilmesidir. Bu ara§tirmanm en 
onemli ara§tirma sorusu, bir sanat simfinda dijital stireg dosyasimn avantaj 
ve dezavantajlan nedir? 

Ara^tirmamn AmacvBu ara§tirmanm temel amaci egitim faktiltesi resim i§ 
ogretmenliginde ogrencinin kendisinin, akramnm ve ogreticinin puanlama 
birle§imi ile dijital stireg dosyasi degerlendirmeyi ara§tirmaktir. 
Ara§tirmanm bir diger amaci ise ogrencinin kendisi, akrani ve ogreticinin 
degerlendirmesi hakkmda ogrencilerin baki§ agilarim ve dtl§tincelerini 
belirlemektir. 

Ara^tirmamn YontemiiBu ara§tirmada, Egitim Faktiltesi Resim I§ 
Ogretmenligi son sinif ogrencilerinden 34 ogrenci tlzerinden veri 
toplanmi§tir. Bu ogrencilerin 14' ti erkek 20' si kiz ogrencidir. Ogrencilerin 
ya§lari 21 ile 28 ya§ arasmda degi§mektedir. Ogrencilerden bilgisayar 
ortammda stireg dosyalanm hazirlamalan istenmi§tir. Stireg dosyalanmn 
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igerigini Ronesanstan guntimtize istedikleri sanatgilari tanitan kesme 
yapi§tirma kitaplarm (scrapbook) olu§turmasi soylenmi§tir. Ogrencilere 
dijital siireg dosyalanni hazirlamalan igin dort haftalik bir sure verilmi§tir. 
Haftalik 8 saatlik uygulamanm 4 saati bilgisayar ortammda yapilmi§tir. 
Ogrencilerin dijital stireg dosyalarmi hazirlamak igin gali§malari stiresince 
Word, Paint Brush, Power Point Presentation, Illustrator ve Photoshop gibi 
bilgisayar programlarmdan ve ge§itli internet sayfalarmdan 
yararlanabilecekleri soylenmi§tir. Ogrencilerin bilgisayar ortammda 
hazirladiklan gah§malari elektronik postalarmda, CD ROM, DVD, 
ta§mabilir bellek gibi ortamlarda depolanmi§tir. Ogrencilerin dergi, gazete 
gibi kagit ortamindaki gah§malari dijital ortama aktarmalari igin tarayici 
(scanner) kullanmalanna izin verilmi§tir. Ogrenciler stireg dosyalanmn 
igerisinde yer alan gah§malarmi (scrapbook) tarihsel bir sira dtizenine gore 
siralayip dijital ortamda sunmu§lardir. Ogrencilerin hazirladiklan siireg 
dosyalari daha onceden belirlenmi§ ve hazirlanmi§ bir yonergeye (rubric) 
gore degerlendirilmi§tir. Ogrenciler ilk olarak yonergeye gore kendi 
gah§malarini daha sonra yine ayni §ekilde yonergeye gore arkada§larimn 
gah§malarini degerlendirmi§lerdir. Son olarak tiim siireg dosyalari 
ara§tirmaci tarafmdan yine ayni yonergeye gore degerlendirilmi§tir. 
Ara§tirmanm nicel boyutunda, ogrencilerin siireg dosyalari igin kendisine 
ve akranma verdigi puanlar ile ara§tirmacimn verdigi puanlarm arasmdaki 
ili|kiye (korelasyona) bakilmi§tir. Ara§tirmanm nitel boyutunda ise 
ogrenciler ile yari yapilandirilmi§ gorii§me teknigi kullamlmi§tir. 
Ogrencilere gali§ma siiresi igerisinde ya§adiklari zorluklar ve kolayhklar ile 
yaptiklan degerlendirme ile ilgili agik uglu sorular sorulmu§tur. 
Ogrencilerin kendi gah§masini, arkada§imn gah§masmi ve ogreticinin 
onlarm gah§malarim degerlendirmeleri hakkmda gorii§leri ahnmi§tir. 
Ogrenciler ile yapilan gorii§meler ses kaydi yapilarak daha sonra metne 
donu§tiirulmu§ ve analiz edilmi§tir. 

Sonuglar ve Tflrh§mfl;Ara§tirmanm nicel sonuglarmda en dil§iik puanlama 
korelasyonu ara§tirmacinm kendisi ile ogrencinin akrani arasmda 
bulunmu§tur. En yiiksek puanlama korelasyonu ise ogrencinin kendisi ile 
akrani arasmda bulunmu§tur. Ara§tirmanin nitel boyutu igin ogrenciler ile 
yapilan gorii§melerde ise be§ temel kategorinin olu§tugu gorulmii§tur. Bu 
kategoriler frekans ve yilzdesi en yilksekten en dil§uge dogru §oyledir: 
Ara§tirmacmin degerlendirmesini onaylama, web sayfalarmin yetersizligi, 
akran degerlendirmede tereddiit, bilgisayar programlarmm zorlugu ve 
kendini degerlendirmede tereddiittur. Ogrencilerin ogreticinin 
degerlendirmesine daha gok giivendikleri gorillmu§tur. Ogrenciler internet 
sayfalarmdan yeterince yararlanamadiklanm belirtmi§lerdir. Internet 
sayfalarmda hazirlayacaklan siireg dosyalari igin sanatgilarm resimlerinin 
bol oldugunu ancak sanatgilar ve eserleri hakkmda yeterince bilgi 
bulamadiklarmi belirtmi§lerdir. Ayrica ogrenciler kesme yapi§tirma 
kitaplanm olu§turmada kullandiklan bilgisayar programlarmda da zorluk 
ya§ami§lardir. Profesyonel bilgisayar programlarmda kiz ogrencilerin daha 
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gok zorlandiklan hatta bu programlari tercih etmedikleri bulunmu§tur. 
Erkek ogrencilerin bazilarmm ise profesyonel programlari tercih ettigi 
gortllmu§tur. Ogrencilerin arkada§inin silreg dosyasmi degerlendirmede 
tereddtit ve endi§eye kapildiklan ogreticinin degerlendirmesinden daha az 
giivenilir olacagini belirtmi§lerdir. Ogrencilerin kendi stireg dosyasmi ve 
arkada§imn siireg dosyasmi degerlendirmede ya§adiklari tereddiidiln Tiirk 
Kiilturu'nden kaynaklandigi du§unulebilir. (^ilnkil Tiirk Kiilturii'nde 
ogretmen ya da ogretici otorite, bilgi sahibi ve daima ogrenciler tarafmdan 
en giivenilir goriilen ki§i olarak du§iinuliir. 

Oneriler: Egitimsel uygulamalar bakimindan bir degerlendirme araci olarak 
dijital siireg dosyasi ozellikle sanatsal gali§malar ile ilgili sinif ortamlannda 
yaygm bir §ekilde uygulanmali ve geli§tirilmedir. Sana! ogretmeni egitim 
programlarmda bu ara§tirmadaki gibi iiglii degerlendirme yakla§imma 
ciddi §ekilde yer verilmelidir. Degerlendirmede yonerge gok onemlidir. 
Degerlendirme yonergesinin kriterleri agik ve net olmali puanlayicilar 
arasmda uyumun dii^ilk olmasmda neden olmamalidir. 

Anahtar Sdzcukler: Sanat egitimi, siireg dosyasi degerlendirme, dijital 
siireg dosyasi, degerlendirme yonergesi 



